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Abstract — Renewable energy sources can be attached to sensor 
nodes to provide energy replenishment for extending the battery 
life and prolonging the lifetime of sensor networks. However, for 
networks with replenishment, conservative energy expenditure 
may lead to missed recharging opportunities due to battery 
capacity limitations, while aggressive usage of energy may result 
in reduced coverage or connectivity for certain time periods. 
Thus, new power allocation schemes need to be designed to 
balance these seemingly contradictory goals, in order to maximize 
sensor network performance. In this work, we study the problem 
of how to jointly control the data queue and battery buffer to 
maximize the long-term average sensing rate of a wireless sensor 
network with replenishment with certain QoS constraints for 
the data and battery queues. We proposed a unified algorithm 
structure and analyze the performance of the algorithm for all 
combinations of finite and infinite data and battery buffer sizes. 
We also provide a distributed version of the algorithm with 
provable efficiency based on imperfect scheduling. 

I. Introduction 

Wireless sensor networks are widely used in monitoring 
[1], maintenance [2], and environmental sensing [3]. The lack 
of easy access to a continuous power source and the limited 
lifetime of batteries have hindered the wide-scale deployment 
of such networks. However, new and exciting developments 
in the area of renewable energy [4] [5] provide an alternative 
to a limited power source, and may help alleviate some of the 
deployment challenges. These renewable sources of energy 
could be attached to the nodes and would typically provide 
energy replenishment at a slow rate (compared to the rate at 
which energy is consumed by a continuous stream of packet 
transmissions) that could be variable and dependent on the 
surroundings. 

Energy management in networks equipped with renewable 
sources is substantially and qualitatively different from energy 
management in traditional networks. For example, conserva- 
tive energy expenditure could lead to (i) instability of the data 
queue because the energy is not being fully used to transmit at 
high enough data rates, or (ii) missed recharging opportunities 
because the battery buffer is full. On the other hand, an over 
aggressive use of energy may lead to lack of coverage or 
connectivity for certain time periods, which could hurt the 
application's requirements. Further, if the battery of a node 
discharges completely, it could be temporarily incapable of 
transferring time-sensitive data to the sink. This may have 
undesirable consequences for many applications. Thus, new 
techniques and protocols must be developed for networks with 
replenishment to balance these seemingly contradictory goals. 



In this paper, we consider single link communication, in 
which the transmitting node has an infinite data buffer that 
holds the incoming variable-rate sensing data and a finite 
battery buffer, which is being replenished at a variable rate. 
One may experience instability of the data queue or frequent 
occurrences of battery discharge (batteries becoming empty) 
if power consumption and sensing rates are not managed 
properly. We investigate the problem of maximizing the long- 
term average sensing rate, subject to data queue stability and 
constraints on the desired rate of visits to zero battery state. 
We provide a simple joint rate control and power management 
scheme, and show that the performance of our scheme is close 
to optimum. 

While the problem of energy management in sensor net- 
works has seen considerable attention, there are relatively few 
works [6], [7], [8], [9], [10], [11] that also include energy 
replenishment. In [6], the authors consider the problem of 
dynamic node activation in rechargeable sensor networks. 
They provide a distributed threshold policy that achieves a 
performance within a certain factor of the optimal solution 
for a set of sensors whose coverage area overlap completely. 
In [7], the authors study the problem of computing lexi- 
cographically maximum data collection rate for each node 
such that no node will be out of energy. In [8], the authors 
consider the problem of energy-aware routing with distributed 
energy replenishment. They provide an algorithm that achieves 
a logarithmic competitive ratio and is asymptotically optimal 
with respect to the number of nodes in the network. However, 
the problems considered in these works is very different from 
the one considered here in that the coupling between the data 
and battery queue is not taken into account. Prior work on 
power allocation for wireless networks without replenishment 
has been widely studied, e.g., [12], [13], [14]. In [12], the 
authors develop approximated algorithms to minimize average 
allocated power, or maximize throughout given average power 
constraint, and at the same time keep the data queue stable. 
The approximation improves at the cost of increasing the data 
queue length and queueing delay. In [13], [14] the authors 
assume that the data buffer is large enough so that packet loss 
does not occur and provide a dynamic programming based 
solution. Most of these works assume a constant energy supply, 
i.e., there is no battery issue. In this paper, we explicitly 
model energy replenishment and consider jointly managing the 
data and battery buffers. This coupling between the data and 
battery buffers is what makes the problem notoriously difficult 
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to solve using standard optimization based approaches. For 
example, unlike prior works that utilize the fact that a static 
allocation policy is optimal, and then develop dynamic policies 
based on their performance against a dynamic policy, it isn't 
even clear whether a static policy would be optimal in our 
setting. Specifically, with a battery buffer, there is an additional 
energy constraint that the allocated energy should be within 
the battery state, and this constraint is even more difficult than 
the average power constraint. 

The main contributions of the paper are as follows: 

• We formulate a sensing rate maximization problem with 
QoS constraints on both data and battery queues. Due 
to the coupling between the battery and data buffers it is 
unlikely that a stationary policy optimal, hence traditional 
resource allocation techniques do not directly apply. 
Further, dynamic programming based solutions result in 
prohibitively high complexity, even for the single-link 
scenario. Nonetheless, we are able to develop a simple 
and unified framework for all combinations of finite and 
infinite data and battery buffer sizes, and our algorithm 
is provably efficient. 

• We extend the algorithm from a single-link case to 
a multi-hop network under node-exclusive interference 
model with practical settings and develop an efficient 
distributed algorithm based on imperfect scheduling. 



II. Single Link Model 

We first consider a single link control model in this paper, 
as illustrated in Figure 1. We assume a time slotted system 
and in time slot t, the amount of data available for the 
sensor node to sense is denoted by A(t), which is upper 
bounded by 0 < A max < oo. In the same time slot, the 
actual amount of data the node senses and places in the data 
buffer is R(t). The amount of energy the node spends at time 
t for data transmission is P(t), which is upper bounded by 
0 < Ppsnk < oo and the achievable data rate at that power 
level is /i(P(t)), where we assume /x(-) to be monotonically 
increasing, reversible and differentiable on the half real line 
5R + U{0}- The node has a battery of size B b (either B b < oo 
or B b = oo) with zero initial battery state, where the energy 
for transmission is coming from and the energy in the battery 
is being replenished at a variable instantaneous rate r(t). The 
energy state of the battery and the data state of the data buffer 
at time t is qb(t) and qd(t), respectively. The data buffer has 
size B d (either B d < oo or B d = oo). Under this setting, we 
describe the objective in the following section. 



A. Problem Formulation 

Our general objective is to maximize the long-term average 
sensing rate, subject to the QoS constraints on both data and 
battery queues: 
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Fig. 1. Single Link Control Model 



1 T_1 

maxliminf — >^ R(t) 
P.R T^oo T ^ w 



s.t. q d (t + 1) = min [(q d (t) - /i(P(<))) + + R(t),B d ] , (a) 
q b (t + 1) = min [q b (t) - P{t) + r{t),B b ] , (6) 
0 < R(t) < A{t), (c) 
0<P(t)<min[ft(t),PpedJ, (d) 
1 T_1 

limsup — ^2 Qd(t) < oo (B d = oo), or 

Pd < rjd (B d < oo), (e) 
Po < Vo, (/) 

where (•)+ = max[-, 0], R = {P(0), P(l), P(T-l), .. .} 

is the actual sensing data vector, P = {P(0), P(l), . . . , P(T— 
1), . . .} is the allocated power vector, and 



T^oo ^ t=0 Kit) 

Po = limsup E ^ io(f) 



(1) 



(2) 



are the long-term data loss ratio with an upper bound rj d , 
and the frequency of visits to zero battery state with given 
threshold r\ Q respectively, where 



D(t) = ((q d (t) - KP(t))) + + R® - Bd) + , 

^o(^) — ^battery state '0' is visited from higher states in slot t 

0 if P(t) = 0 or P(t) < q b (t) 

1 otherwise 



(3) 



(4) 



are the amount of data loss in slot and the indicator that the 
battery discharges completely in time slot t, respectively. Note 
that, if the battery is completely discharged at time t and there 
is no replenishment at slot t, we do not consider a complete 
discharge event occurring at time t + 1 as well. We do not 
assume ergodicity of the system parameters, but if they are 
ergodic, then p Q represents the actual probability of a complete 
discharge event as t — * oo. 

In Problem (A), constraints (a) and (b) describe how the data 
and battery queues evolve, respectively. Especially, if B d = 
oo, (a) can be simplified to 

q d (t + 1) = (q d (t) - KP(t))) + + R{t), (a 1 ) 
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and if B\> = oo, (b) can be simplified to 

q b (t + l)=q b (t)-P(t) + r(t). (&') 
Constraint (c) bounds the actual amount of sensed data R{t) by 
the available amount of data A{t) in slot t. Constraint (d) states 
that we cannot oversubscribe the energy that is unavailable in 
the battery nor can we exceed the peak power level. Constraint 
(e) is the QoS constraint for data queue: if Bd = oo, we need 
to keep the data queue stable, and if Bd < oo, the data loss 
ratio is required under a given threshold. Constraint (f) is the 
battery QoS constraint of the desired battery discharge rate r\ 0 . 

Problem (A) has an inventory control structure, and typically 
such a structure can be solved optimally using dynamic 
programming, albeit with high complexity. Furthermore, de- 
pending on the exogenous processes {A(t), t > 0} and 
{r(t), t > 0}, Problem (A) may not have a feasible solution, 
i.e., there exists no power allocation policy {P(t), t > 0} 
that satisfies all constraints simultaneously. In this paper, our 
purpose is to develop a simple algorithm, which performs 
arbitrarily close to the performance of the optimal power 
allocation, whenever a feasible solution exists. To achieve that 
purpose, 

• we define virtual queues for both the data and battery 
buffer to avoid the difficulties involved in dealing with 
the data loss and battery discharge probability directly. 
We show that keeping the virtual queues stable ensures 
that the constraints on pd and p 0 are met. 

• we design an power allocation scheme based on simple 
index policies and show that our scheme keeps both 
virtual queues stable and at the same time performs 
arbitrarily close to the optimal performance. 

• we generalize our algorithm to the multihop scenario and 
develop distributed algorithms. 

B. Virtual Queues 

We define q~d and q b as the virtual data and battery queues. 
The virtual queues evolve according to the following Lindley's 
queue evolution equations: 

q d {t + 1) = (q d (t) - VdR(t))+ - p{P{t)) + R{t) + J(i), 

(5) 

q b (t + 1) = (q b (t) - Vo )+ + P(t) - r(t) + M(t) + l a (t), 

(6) 

where I(t) — (p(P(t)) — qd(t)) + is the amount of sent idle 
packets when there is no enough data to transmit using the 
allocated energy, M(t) = (q b (t) - P(t) + r(t) - B b ) + is the 
amount of missed replenishing energy due to full battery when 
B b < oo, and l 0 (t) is defined in Equation (4). Note that if 
B b < oo, M(t) = 0 and Equation (6) reduces to 

q b (t + 1) = (q b (t) - 7] 0 ) + + P(t) - r(t) + l 0 (t). (7) 

Without loss of generality, the initial state q b (0) and q b (0) 
can be set to be zero. The following proposition shows that 
if the virtual queues qd(t), q b (t) and the actual battery queue 
q b (t) are all strongly stable, pd and p a are guaranteed to meet 
their constraints. 



Proposition 1: If the virtual queues q d (t), q b {t) and the 
actual battery queue q b (t) are both strongly stable, i.e., 




then p d < rid and p 0 <rj 0 . 

The proof of this result can be found in Appendix A. Next, 
we present our scheme. 

III. Joint Power Allocation and Admission 
Control Algorithm 

A. Algorithm 

The algorithm consists of two components: a rate control 
component and a power allocation component. Both compo- 
nents are index policies, i.e., the solutions are memoryless 
and they depend on the instantaneous values of the system 
variables. 

Rate Control (RC): 
We define 0 < V < oo to be the control parameter of our 
algorithm. Let Qd(t) = qd(t) when B d = oo, and let Q d (t) = 
q~d{t) when Bd < oo. If Qd(t) < y, the transmitting node 
chooses to sense all the available data packets, i.e., R(t) = 
A(t). Otherwise, R(t) = 0. 

Power Allocation (PA): 
Solve the following optimization problem 

max Q d {t)ti{P{t))-q b (t)P{t), (8) 
P(t)en(t) 

where U(t) = {P(t) : 0 < P(t) < mm[q b (t), P peak ]} is a 
compact and nonempty set. Allocate P(t). 

The set, H(t) of possible power allocations guarantees 
Constraint (d) on P(t) in Problem (A). If /i(-) is concave, the 
objective function is a concave function of P(t). Consequently, 
PA solves a simple convex optimization problem in each time 
slot. The positive term Qd(t)p(P(t)) can be viewed as a utility 
of power allocation P(t) and the term q b (t)P(t) can be viewed 
as its associated cost. When the control parameter V is chosen 
to be large, Q d (t) tends to be large according to RC, and 
PA tries to allocate higher power P(t) to increase the utility, 
whereas, when the virtual battery queue length q b (t) is large, 
PA avoids allocating a high amount of power to reduce the 
cost. Thus, this index policy of PA can be viewed as a greedy 
profit maximization scheme. 

B. Performance Analysis 

Recall that A(t) is the available amount of sensing data and 
R(t) is the actual amount of data the transmitting node chooses 
to sense. Clearly, using the rate controller, we make sure that 
the data queue remains within a certain bound and this has a 
positive effect on the battery as well, since certain proportion 
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of the data packets are not allowed into the transmitting node. 
The natural question one would ask here is, whether our 
rate controller rejects too many packets in the first place to 
synthetically meet the constraints. In the following theorem, 
we show that this is not the case. Indeed, if there exists a 
solution, A* = liminf T _ ) . 00 ^ J2t=o ^*(*) t0 Problem (A) 
for the arrival process {A(t), t > 0} and the replenishment 
process {r(t), t > 0}, then the sensing rate associated with RC 
and PA can be made asymptotically closer to A* by increasing 
the control parameter V with increasing Bd and B b . We use 
the notation y — O(x) to represent y going to 0 as i goes to 
0. 

Theorem 1: If 

(1) /u(-) is concave on 3?+ U{0}> an d i ts sl°P e at 0 satisfies 1 

0 < 0 = m'(0) < oo, 

(2) r(t) > 0, Vi > 0, and f = limsupy^ £ J2l=a r ( t ) < 

Rpeaki 

(3) A feasible solution to Problem (A) exists and the optimal 
instantaneous sensing rate is R*(t), 

then the joint power allocation and admission control algo- 
rithm (with RC and PA) achieves: 



Q d {t) < 



v 



V t > 0 



,v 



%{t) </?(y + 

q b (t) < oo, V t > 0 

T-l , T-l 



(9) 

V t > 0 (10) 



(11) 



liminf - V > liminf - V R*(t) - O(-) 



t=0 



t=0 



-??o(/"max + /3)-5(V,B d ,B 6 ), (12) 



where ^i max = ^(P pea k) is the upper bound for the transmission 
rate, and 



g(V,B d ,B b ) = { 



), 



0{ 
0( 



v , 

-B d ) + 



(fy-B 6 )+ 



if B d < oo, B b < oo 
if Bd = oo, B b < oo 
if Bd < oo, B b = oo 



) + o( 



), 



if < oo, B b < oo. 



The proof of Theorem 1 can be found in Appendix B. From 
Equation (9), Equation (10) and Equation (11), the virtual 
queues qdif), q b (f) and the actual battery queue q b (t) are 
all strongly stable. Thus, by Proposition 1, pd < i]d and 
Po < Vo- In Theorem 1, V is a finite tunable approximation 
parameter that controls the efficiency of the algorithm. Observe 
Equation (12), the term ?7 0 (^ max + (3) captures the influence 
of battery outage, and it is small since the battery outage 
threshold r) 0 is usually set to be very small to avoid network 
disconnection. g(V, Bd, B b ) represents the asymptotical prop- 
erty of the gap, i.e., if at least one of the buffer size is finite, 
as V increases, the buffer size should increase accordingly to 
keep the gap small. 

1 For instance, consider the additive white Gaussian noise channel capacity, 
/i(P) = log(l + P/No), where No is the two sided noise power spectral 
density. 



IV. Joint Energy and Queue Management in 
Multihop Networks 

A. Formulation 

We consider a multihop wireless sensor network with N 
nodes and L links. Each node n € N = {1,2, ... ,N} is 
attached to power sources for replenishment. Let A^ (t) and 
Rn(t) denote the amount of available data for sensing and 
the actual amount sensing data, to node n that are destined 
to node e in slot t. Assume that each node n maintains a 
infinity data buffer with state q^ e {t) for flows destined to e, 
and also maintains a finite battery buffer with size B^ and 
state q b (t). Let r n (t) denote the replenishment at node n in 
time slot t. The transmit power is chosen to be Pi (t) over link 
/. In the formulation, we assume that the power the receiving 
node consumes to receive and decode the packet is identical 
to Pi (t) as well. The sole reason for this is simplicity and the 
generalization to the asymmetric case is straightforward. We 
use the node-exclusive interference model. Under this model, 
a node can only receive from or transmit to at most one node 
at any time slot, and in each time slot t, with the assigned 
power Pi(t), the achieved data rate at link I is ni(Pi(t)) in 
that time slot, where the rate function m(-) is a non-decreasing 
and differentiable function satisfying yU;(0) = 0. Let p™ be the 
frequency of visits to the zero battery state for node n. Let 
fi„ and 6„ denote the set of directed links originated from 
node n and terminate at node n, respectively. In a multihop 
network under node-exclusive model, we formulate the joint 
queue and energy management problem as follows: 



T-l 



(B) 



maxlhninf^^ ^ R c n (t) 



P.R 



t=0 n,e£N 



s.t. P(t) satisfies the node-exclusive model, 

C e (t + 1) = (C e (t) E rt(W))) + + K(t) 

ien n 

+ tf(W)), ri^e, 



(a) 



(b) 



iee„ 



q b {t+l) = min 



tf(t)- ]T Pi(t) + r n (t),B t " ,(c) 
zen„ue„ 

0 < ]T Pi(t) < q£(t), 0 < Pi(t) < P peak , (d) 
ieo„ue n 

N 

5>f(i}(*)) - Mm)), Kit) < A e n (t), (e) 



e=l 



T-l 



limsup I- 9d' 6 W < °°> n^e, 



t=0 



(/) 

(9) 



where 77™ is the desired upper bounds for p™, P{t) is the 

power assignment vector for all links in slot t, P is the power 
assignment for all links over all time slots, and R is the actual 
sensing data vector for all node-destination pairs over all time 
slots. In Problem (B), the objective is to maximize the long- 
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term average total sensing rate to all nodes destined to all 
destinations. 

In Problem (B), (a) is the interference constraint, constraints 
(b) and (c) describe how the data and battery queues evolve, 
respectively. Note that the destination node of each flow 
does not need to maintain a data buffer for that flow, as 
indicated in (b). Constraints (d) are the energy conservation 
equations stating that we cannot oversubscribe the energy that 
is unavailable in the battery nor can we exceed the peak power 
level. Constraints (e) are the rate conservation equations that 
bound the actual amount of sensed data R(t) by the available 
amount of data A(t), and share the transmission rate of a 
link among all the destinations in slot t. Constraint (f) is the 
QoS constraint for data queue: we need to keep all the data 
queues stable. Constraint (g) is the battery QoS constraint of 
the desired battery discharge rate r/ 0 . 

Similarly, we define virtual queues for all n E ft 

#(t + l) = (g?(t)-»£) + + £ W)-r n (t) 

iefi„ue„ 

+ M n (t) + r:(t), (13) 

where M n (t) = (q£(t) - £* e n n ue n fl(t) + r„(t) - B£)+ is 
the amount of missed replenishment and 

7"(i) = indicator that battery state '0' is visited from higher 
states in slot t for node n 

0 if E ief2 „ u e„ P W = 0 or £*en„ U e„ P i (*) 
< «?(*) 

1 otherwise 

Next, we generalize the main results and algorithms RC and 
PA to the multihop scenario. 

Corollary 1: If all the virtual battery queues q% (f), VneN 
are strongly stable, we have p™ < rf£, Vn e ft. 
The proof is identical to the single hop scenario and can be 
found in Appendix C. We give the algorithm in the following 
section. 

B. Algorithm 

Multihop Rate Control (MRC): 

Maximum Weighted Matching Algorithm: If q^' e (t) < \ , 
node n chooses to sense all the available data packets, i.e., 
Rn(t) — Affi); otherwise, reject all the arrivals, i.e., -fi^(i) = 
0. 

Maximal Matching Algorithm: If q^' e (t) < V , node n 
chooses to sense all the available data packets, i.e., -fi^(i) = 
A^(t); otherwise, reject all the arrivals, i.e., R-n(t) = 0. 

Multihop Power Allocation (MPA): 
First define 



ll 



7 if rec(Z) ^ e 
0 otherwise 



where 7 > 0 is some constant for the sake of the algorithm. 

t . /,\ r tran(l).e rec(/),e 

Let ei(t) = argmax e {g d ~Q d 
that link I puts all the rate, and wi(t) — max[q z J 

rec(l) .ei (t) / ,\ ei(t) ~n . ^ . 

Id 



— 7f } be the flow on 

tran(l),ei(t) _ 



backlog. This means that no node can transfer a flow to a 
relay node that is not the destination of the flow unless the 
differential backlog of the data queues between these two 
nodes is no less than a fixed value 7. Here, tran(l) and 
rec(l) denote the transmitting and receiving node of link I, 
respectively. 

For each link I, solve 



max W*)WW*)) - {% an[l \t)+% C{l \t))Pm 

(14) 



where II; (i) = 



tran(l) / ,\ 
% (t), 



{W) 

the solution. With the 

Wi(t) = wi(t)m(Pi(t)) 

be the weight on link I. 



< 



Let 



calculated 

■ (C n(l \t) 



W) < 

Pi(t) be 
let 



power Pi(t), 

€ c{l \t))W) 



rec(0,e,(i)^ _ 7 e,(t) )0 j [& related tQ ±Q modifie d differential 



For the whole network, 
Maximum Weighted Matching Algorithm: link I has weight 
Wi{t), then the weight of a matching M is Ww(t) = 
"Yl,i e jftWi(t). The network chooses a maximum weighted 
matching in a centralized manner, the links in the cho- 
sen matching become active with the calculated transmitting 
power, and other links are not activated. 

Maximal Matching Algorithm: the network chooses a maximal 
matching in a fully distributed manner as in [15], the links 
in the chosen matching become active with the calculated 
transmitting power, and other links are not activated. 

Routing: 

When wi (t) > 0, transmit for flow that is destined to e; (t) with 
rate i.e., M^W)) = WWW) and /if(fl(t)) = 

0, Ve^e;(t). 

MAC and routing can be done by each node independently. 
Note that MPA has two different algorithms, the maximum 
weighted matching based algorithm is centralized and the 
maximal matching based algorithm is fully distributed. We 
give our main theorem for the multihop scenario: 

Theorem 2: If 

(1) fii(-) is concave on 3? + 1J{0}, and its slope at 0 satisfies 

0 < P = mK°) < 00, yi e L, 

(2) Vn e ft": r n (t) > 0, Vi > 0, and f n = 

liniSUpy^ ^ J2j=0 r n(t) < -Ppeak, 

(3) A feasible solution to Problem (B) exists and the optimal 
instantaneous sensing rate vector is R*(t), 

then the maximum weighted matching based joint rate 
control MRC, power allocation MPA, and routing algorithm 
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achieves: 



Replenishment Distribution 



qY{t) < j + A max , 

#)</3(y + ^ax), 



> £ { liminf ^ 2 E <W ~ ^ 



t=0 e 
T-l 



'0 



0) 




t=0 e 

•n 

0(11- "6 



(17) 



Further, if the variable r(t), Vt > 0 are independent from slot 
to slot, and the variable A(t), Vt > 0 are also independent 
from slot to slot, then the maximal matching based joint rate 
control MRC, power allocation MPA, and routing algorithm 
achieves: 



> 




q n d ' e {t)<V + A mzx , 



(18) 
(19) 



lim inf — 



■0) 



t=0 e 

v 



) 



(20) 



The proof of Theorem 2 can be found in Appendix D. The 
results can be interpreted similarly as Theorem 1. 

V. Numerical Example 

In the simulation, we consider a network topology as shown 
in Figure 2 (a). There are 6 nodes, 7 links, and 2 flows 
with source-destination pair (3, 1) and (5, 2), respectively. 
The number of time slots is T = 10 6 , and the duration 
of each slot is 10 sees. We use the rate power function 
Hi{Pi) = 101og 2 (l+ a j^ L ) packets/slot V/ € £. Let the power 
of the background noise Ni = 1.6 x 10~ 14 W, V/ £ L, and 
the channel gains gi = 1.6 x 10 -13 , V/ G XL. Each node is 
equipped with an infinite data buffer for each flow through 
it, and a battery buffer of size = 800J. The numbers of 
arrivals A^(t), t > 0, for all nodes and flows, are independent 
Poisson random variables with mean 20 packets/slot and 
A m ax = 31 packets/slot. The replenishment process is periodic 
with independent Gaussian noise, as shown in Figure 2 (b) 
(The cycles imitate the daily solar cycles for a solar battery). 
We also set ry™, the threshold of battery outage probability to 
0.03 for all n € N. 

We simulate the proposed algorithm for different values of 
the control coefficient V and compare the results with the 
optimal value 2 . From Figure 3 (a), we see that as V increases, 

2 The optimal value can be obtained by using dynamic programming. The 
details are omitted here. 
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Fig. 2. Simulation Setting 

the average total sensing rate keeps increasing and gets closer 
to the optimal value, which is consistent with Equation (17). 
From Figure 3 (b), we see that as V increases, the average data 
queue length (we here only plot the data queue length of node 
2 for flow 1) keeps increasing but is upper bounded by the 
bound we get in Equation (15). From Figure 3 (c), the battery 
discharge probability (we only plot for node 1) increases to 
the threshold as V increases. Thus, increasing V can increase 
the average total sensing rate, but the cost is increasing the 
average data queue length and battery discharge probability 
of at least one node. 






Fig. 3. Performance of the MWM based algorithm. Impact of the control 
parameter V on (a) the average total sensing rate, (b) average data queue 
length, and (c) the battery discharge probability 



Similarly, from Figure 4 (a), we see that as V increases, 
the average total sensing rate keeps increasing and gets closer 
to some value that is larger than half optimum, which is con- 
sistent with Equation (20). From Figure 4 (b), we see that as 



7 



V increases, the average data queue length (we here only plot 
the data queue length of node 3 for flow 1) keeps increasing 
but is upper bounded by the bound we get in Equation (18). 
From Figure 4 (c), the battery discharge probability (we only 
plot for node 6) increases to the threshold as V increases. 





Q 



m u 
E 



V 
(b) 



(c) 



Fig. 4. Performance of the MM based algorithm. Impact of the control 
parameter V on (a) the average total sensing rate, (b) average data queue 
length, and (c) the battery discharge probability 



VI. Conclusion 

In this paper, we studied the problem of energy management 
in rechargeable wireless sensor networks. Our objective was 
to maximize the average data sensing rate subject to QoS 
constraints on both data and battery queues. We provided a 
simple and unified framework of joint rate control and power 
allocation for all combinations of finite and infinite data and 
battery buffer sizes. We showed through both analysis and 
simulation that the performance of our strategy is close to 
that of the optimal solution. We extended our algorithm to 
the multihop scenario and showed that simple extensions of 
our index schemes for single hop generalize to the multihop 
scenario with a similar, close-to-optimal performance. We 
developed a distributed joint rate control, power allocation and 
routing algorithm for multihop networks under node-exclusive 
interference model based on imperfect scheduling. 
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Appendix 

A. Proof of Proposition 1 

Using the idea similar to [12], we have the fact that 
if any queue represented with Q(t) is strongly stable, 
then limsupy^^ ^P- < 0. Hence, if qd{t), and 



?d(T) 
T 

gb(T) = Q 



0, 



qb(t) are strongly stable, then limsup T _ 
limsupj^^ ^4p- < 0 and limsupy^^ 

From Equation (5), we have qd(t + 1) > qd(t) — rjdR(t) + 
D{t) - D(t) - (i(P(t)) + R(t) + I(t). Note that q d (t + 1) = 
q d it) - Ai(P(t)) + I(t) + R(t) - D(t). By adding from 0 to 
T — 1, dividing by T and taking limsupj^^ on both sides, 
we have 

r Qd(T) . ,. <7d(0) 1 ^ 

hmsup — — > lim — rid limsup - > R(t)+ 

T^oo 1 T^oo 1 T— oo 1 frj 

q d (0) - q d (T) 



1 T_1 

limsup— y D(t) + lim 

1 — oo t=0 



T 



Since lim sup T _ 

->oo 7^ — 



linii 



0 and liniT 



< 0, limT_ 

1d(T) 

OO rp 



gd(0) 



0, 



0, so we get 
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Similarly, from Equation (6), we have q b (t + 1) > q b (t) — 
r) 0 + P{t)-r{t)+M(t) + l 0 {t). Note that q b (t + l) = q b (t) - 
P(t) + r(t) - M(t). By adding from 0 to T — 1, dividing by 
T and taking limsupy^^ on both sides, we have 

,. Qb(T) q b (0) 1 ^ 

limsup — — > hm — r] 0 + limsup - > 7 0 (t) 

T^oo T 

Since limsup^^ < 0, limr^ ^5) = 0 , 

liniT^oo ^ = 0 and lim-r^ = 0, so we get 

Po = lim supy^ i J2l=o ] o(t) < Vo- ■ 

B. Proof of Theorem 1 

Proof of Equation (9): Note that I(t) < fi{P{t)) and 
R(t) < A max . The rate allocation unit RC is chosen to satisfy 
Equation (9). 

Proof of Equation (10): Since ^(-) is concave on 3?+ U{0}> 
we have n(P(t)) < fj,(0) + 0P(t) for P(t) G U(t), Vi > 0, 
where 0 < /3 = ju'(0) < oo. Then, Q d (i)/x(P(i)) - 
q b {t)P{t) < Q d {t)n{0)+PQ d {t)P(t)-q b {t)P{t) where P(t) 
is the solution of PA. 

If PQ d {t)P{t)-q b {t)P{t) < 0,thenwegetQ d (t) M (P(t))- 
q b (t)P(t) < Qd(i)/i(0). However, PA chooses P(i) that maxi- 
mizes Q d {t)n(P(t))-q b (t)P(t) which means Q d (t)n(P(t))- 
q b (t)P(t) > Q d (t)n(0) since 0 e U(t). Then we must have 
PQd{t)P{t) - q b (t)P(t) > 0, i.e., pQ d {t)P{t) > q b (t)P(t). 
Thus, if P(t) > 0, < f3Q d {t) < + A max ); if 

P(t) = 0, by Equation (4), l 0 (t) = 0, and we also have 
M(t) < r(t), then q b (t) does not increase anyway. Therefore, 
q b (t) < /3(-|- + A max ) for all t, which is Equation (10). 
Proof of Equation (11): Equation (11) is trivial when B b < 
oo. We only need to consider the scenario when B b < oo. 
Without loss of generality, let ^(0) = 0. We have the 
following cases: 

i) if P(t) > r(t), l a (t) = 0 and q b (t) > 0, then q b (t + 1) < 
q b (t) and q b (t + 1) - q b (t) < q b (t) - q b (t + 1), i.e., even if 
q b (t) increases, the increment is no larger than the decrement 
of q b (t); 

ii) if P(t) < r(t), l a {t) = 0 and q h (t) > 0, then 
q b (t) — q b (t + 1) > q b (t + 1) — q b (t), i.e., the decrement of 
q b (t) is no less than the increment of q b (t)\ 

iii) if l 0 {t) = 1, then q b (t + 1) = r(f) < r max by definition of 
discharging event Equation (4); 

iv) if q b (t) < 0, by Equation (8), PA chooses P{t) = 
min[q> 6 (i),P p eak], then either q b (t + 1) = r(t) < r max , or 
q b (t + 1) = — Ppeak + K*)- F° r me latter case, if 
Ppeak > f(t), then the battery queue state decrease and 
q b {t+l) = q b (t)-q b (t+l) = P pea k-r(i) > 0; if P peak < r(t), 
then q b (t) < 0 and case iv) continues. However, P pea k > r, 
which means after at most K < oo slots, r(t + K) < P pea k 
and the battery queue state starts to decrease. 



Combine the above discussion with Equation (10), we have 
that Vt > 0, there 3t 1 (t),t 2 {t) > 0 such that 

q b (t) < min[ gfc (0),r max ] + \{q b {h (*)))+ - {q b {t 2 {t)))+\ 

-\- K{v milx Ppeak) ^ 

v 

<(K + l)r max + (3(- + A max ) < oo, 
which is Equation (11). 

Proof of Equation (12): We define the Lyapunov function 

L(Q d (t),q b (t)) = Q 2 d (t) + ql(t), and A(Q d (t), q b (t)) = 
L(Q d (t + l),q b (t + 1)) - L(Q d (t),q b (t)). 
I) B d = oo and B b = oo. 

From Equation (7), we have q£(t + 1) < (q b (t) — r] 0 ) 2 + 
(l a (t) + P(t) - r(t)f + 2(q b (t) - V o) + (lo(t) + P(t) - r(i)). 
Also from the data queue dynamics, we have q\{t + 1) < 
qj(t) + M 2 (P(t)) + R 2 (t) + 2q d (t)R(t) - 2q d (t)fx(P(t)), then 

A = A(q d (t),q b (t)) 

< v 2 (P(t)) + R 2 {t) + 2q d (t)R(t) - 2q d (t)n(P(t)) + (1+ 
^peak) 2 + r, 2 nax + r, 2 0 + 2?7 0 r max + 2q b (t)(l 0 (t) + P(t)- 
r(t)) 

< (1 + Ppeak)' + Vl + rL, + 2?? 0 r m ax + Mm a x + ^max(^) + 

VR(t) + 2q b (t){l 0 {t) - r(t)) + 2 [(,«,(*) - V/2] J2(t)- 
2 [ % (t) M (P(t)) - . 

It is apparent that RC is trying to minimize the term 
[qd(t) — V/2] R(t), and PA is trying to maximize the value 
of the term [q d (t)/j,(P(t)) - q b (t)P(t)]. Since the optimal 
solution for Problem (A) may not be unique, we let T* 
be the optimal solution set and P* e be any optimal 
solution, for Problem (A). In time slot t, let P m (i) be 
the value that maximize the unconstrained objective func- 
tion q d (t)n(P(t)) - q b (t)P(t). Since n(t) = {P(t) : 
0 < P(t) < mm[q b (t), P pea k]}, only when q b (t) < P pe ak 
and P m (t),P*(t) i n(t), we may have q d (t)fi(P(t)) - 
q b {t)P{t) < q d {t)fj,{P*{t)) - q b (t)P*(t). Thus, we have 

A < (1 + Ppeak)' + Vl + i + 2r/o W + mLc + ^max + V{ 

R(t) - R*(t)) + 2q d {t)[R*{t) - n{P*{t))] + 2q b {t)l a {t) 
+ 2q b (t)[P*(t) - r(t)] + 2[q d (t)(n(P*(t)) - fi(P(t))) + 
q b (t) (P(t) - P* (f))]7[p mW) p. (t)^n(t)] n [9b(t)<P P cak] ■ 

When P m (t) i n(t) and g 6 (t) < P peak , PA will allocate 
<7b(t) amount of energy for transmission in order to maximize 
q d (t)n(P(t))-q b (t)P(t) within IL(t). Under this situation, we 
must have l 0 (t) = 1 since r(i-l) > 0. Further, P*(t) £ Ii{t) 
means P*(t) > P(t). Thus, 

A < (1 + Ppeak) 2 + vl + r 2 max + 

R(t) - R*(t)) + 2q d (t)[R*(t) - n(P*(t))] + 2q b (t)[P*(t) 
- r(t)] + (V + 2A m , x ){(3 + Mmax)io(t). (21) 

Lemma 1: There exists an optimal policy, under 
which the actual battery queue is strongly stable, i.e., 

limsupr^ ^ *£f=o < 00 • 
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Proof: Suppose there is an optimal policy under which 

lim supy^^ 7^ J2t=o ll (*) = 00 • This means there exists 
a subsequence of times {t n } such that t n — > oo and 
^ E^o 1 <3fc (*) oo- In other words, VQ > 0, 3t N e {t„} 
such that Vtfe G {t„} and t fc > ijv, we have ^- J^t'Lo 1 Q b (t) > 

Q. 

Let N(i) = min{n : ± r 0) > ^} and N = 

maxj = o,i,2,... N(i). Note that N < oo, otherwise it contradicts 
the fact that liminfT^oo ^ Y^t=o r (t) = ^ smce r(t) < 

^max < OO, Vt > 0. 

Find Q and the associated t N such that ^- Et^o" 1 9j> W > 
Q > Nr. Since tjv < oo, 30 < T 0 < t/v - 1 such that 
ql(T 0 ) > Q > Nr. Then, if we modify this optimal policy 
by letting P*(t) = f < P pe ak, Vi > T 0 , and keeping P*(t) 
unchanged \/t < Tq, we will not violate the constraint on the 
battery discharge. Under the modified policy, we have 



T-l 



=o 

r £ M (P*(f))+5>(r) 



T^oo T 

= liminf — 

T^oo T 



T-l 



t=0 



t=T 0 



= Kr) (22) 



Further, by Jensen's inequality, we have 



1 T-l 1 T-l 

lim sup - M(i"(t)) < M(limsu P - ]T P*(i)) 



T^oo 



/-(I T^oo t _ Q 

T-l 



< ^(limsup }- r(t)) = n{r), 



T^oo 



t=0 



which means fi(f) is the maximum achievable service rate. 
Combined with Equation (22), the modified policy can provide 
the maximum service rate, so with the original optimal sensing 
rate, the data queue is still strongly stable. Thus, the modified 
policy is still optimal. 

Under the modified optimal policy, ql(t) < r max To < 
oo, Vi < T 0 . Let M(i) = min{n : £Eti" _lr (*) < 
r} and M = max i= o,i,2,... M[i) < oo. Then q* b {t) < 
^maxTo + M(r max - f) < oo, Vi > T 0 . Thus, the battery 
queue is strongly stable under the modified optimal pol- 
icy, i.e., limsupy^^ ^J2t=o < oo. As a byproduct, 
limsup^^ ^jP- = 0. ■ 

Note that q£(t + 1) = q b (t) - P*(t) + r(t) for the optimal 
policy. By multiplying q b (t) for both sides and rearranging 
terms, we obtain q b (t)[P*(t)-r(t)} = q b (t)[q* b (t) - q* b (t+l)}. 
Further, -r max < q b (t) < /3(^ + A max ). With Lemma 1, 
by summing from 0 to T — 1, dividing by T and taking 



liminfr^oo, we have 



T-l 



liminf ^Y,^t)[P*(t)-r(t)\ 

1 — *oo 1 ' 



t=0 



= liminf g b (oH(Q) - UT)ql(T) 



T^oo 



1 

_/ — >oo 1 * — ' 

t=l 

1 T 

< (1 + Ppeak) lim inf - ^ (t) , (23) 



T^oo T ■ 

t=l 

which is a finite constant that is not related to V. 

Note that q*(t + 1) = (q* d (t) - n(P*(t)))+ + R*(t) > 
<j* l (t)-fj,(P*(t)) + R*(t). By multiplying both sides with q d {t) 
and rearranging terms, we obtain qd(t)(R* (t) — /i(P*(i))) < 
qd{t)(q* d {t+l)-q d (t)). With the optimal policy, q* d is strongly 
stable. By summing from 0 to T— 1, dividing by T and taking 
liminfT^oo, we have 

T-l 

liminf = J2q d (t)[R*(t)- »(P* (t))] 
— t — 0 
< lim inf g d (T) g S(r)-g d (0)g^(0) 

- T^oo T 



1 

T^oo 1 ^ 

1 T 



T^oo T 

t=l 

which is a finite constant that is not related to V. 

By summing from 0 to T — 1, dividing by T and V, taking 
lim infT^oo over Equation (21), combined with Equation (23), 
and Equation (24), we get 

^ T-l ^ T-l 

h > inf r E ^ h > inf r E R * (*) - + ^) 



t=0 



t=0 



T^oo T 

II) B d = oo and < oo. 

FromEquation (6), we have q b (t+l) < (qb(t)--q 0 ) 2 + (l 0 (t) + 
P(t) - r(t) + M(t)) 2 + 2(q b (t) - Vo) + (lo(t) + P(t) - r(t) + 
M(t)), then 

A = A(q d (t),q b (t)) 

< (1 + Ppeak) 2 + Vl + r 2 m . dx + 2 Vo r m . ix + ^ 2 {P{t)) + R 2 {t) 
+ VR(t) + 2 [q d (t) - V/2] R(t) - 2[q d (t)fi(P(t))- 



q b {t)P{t)] + 2q b (t)(l 0 (t) - r(t) + M{t)) 



< (1 + Ppeak) 2 



2 

max 



K(P(i) - «*(*)) + 2%(i)[i?*(i) - li{P*(t))\ + 2gfc(t)[ 
P*(t) - r(t) + M*(i)] + (y + 2A max )(/3 + A/max )7 0 (i) 
+ 2q b {t)M{t) 

Without loss of generality, let q b (0) = 0. We have the 
following cases: 
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i) if P(t) > r(t), l a (t) = 0 and q b (t) > 0, then M(t) = 0, 
q b (t + 1) < q b (t) and q b (t + 1) - q b {t) < q b (t) - q b {t + 1), 
i.e., even if q b (t) increases, the increment is no larger than the 
decrement of q b (t); 

ii) if P(t) < r{t), l 0 (t) = 0 and q b (t) > 0, then 
q b {t)-q b {t+l) = r{t)-P{t)-M{t)- Vo + {j lo -q b {t))+ > 
q b (t + 1) - q b (t) = r(t) - P(t) - M (t), i.e., the decrement of 
q b (t) is no less than the increment of q b (t); 

iii) if l 0 (t) = 1, then q b (t+l) = min[r(i), B b ] by definition of 
discharging event Equation (4). If M(t) > 0, then this means 
r(t) > B b and -r{t) + M{t) = -B b ; 

iv) if q b {t) < 0, then q b (t)M(t) < 0 anyway. 

Combine the above discussion with Equation (10), we have 
that Vt > 0, if M{t) > 0 and q b (t) > 0, we must have 
Qb(t) < /3(f + A mnx ) + max[r max , 1] - B b . Thus, 



Note that only when q d (t) < /i max , I(t) may be strictly 
positive. Then combine with the above discussion, we have 

T— 1 



< 



V T- 



t=0 
Mmax 



Bd) + Mn 



V 



0( (i_M- 



T-l 



< 



t=0 

(^(f + Anax) + max[r max , 1] - B b )+r n 



Construct an auxiliary queue q*(t) with the following 
evolution: 

q*(t + 1) = g*(t) - (VdR*(t) + e) + D*(t), 

where e > 0 can be arbitrarily small. Since for an optimal 
policy, 

T-l T-l 

limsup - D *W < %liminf - ]T 



*=0 



t=0 
T-l 



(fv^-^) + 

V 



< 



lim sup I- (VdR* (t) + e) , 



)• 



t=o 



The remaining argument is similar to case I), and we obtain 

?7oOmax + /?) 



. T-l . T-l 



so (?*(£) is strongly stable. 

Note that &(*)[!>*(*) -%i2*(t)] - «d(t)[?*(*+l)-?*(t) + 
e]. By summing from 0 to T — 1, dividing by T and V, and 
taking liminfV^oo, we have 



t=o 



t=o 



T-l 



V 



\ x Tj£ \ £ Ut) [D * {t) mR * {t)] 



III) Bd < oo anc/ = oo. 

From Equation (7), we have q b (t+l) < (q b (t)-r] 0 ) 2 + (l a (t) + 
P(t)-r(t)) 2 + 2(q b (t)- Vo )+(l 0 (t) + P(t)-r(t)). Also from 
Equation (5), we have q 2 (t + 1) < (&(*) - %#W) 2 + (J(t) + 
R(t)-f,(P(t))) 2 +2(q d (t)- Vd R(t))+(I(t)+R(t)-v(P(t))), 
then 



V t 
< — lim inf 

V T^oo 



q d (T)q*(T) 



gd(0)g*(0) | grfW e 



1 1 



T/ T- 



V 



t=i 



A = A(%(i),g 6 (i)) 

< (1+Ppe ak ) 2 + 

• i? 2 (t) + yi?(t) 



1 1 

< T7^max Inn inf - ^ (f ) 



7-Lx + 2r/ 0 r max + M 2 (^(i)) + (1 + Vd) 
2q b {t){l 0 {t) - r(t)) + 2q d (t)I(t) + 2 



V 



*=i 



( 2 + ^max) 



The remaining argument is similar to case I). Further, by 

[(1 - m )q d {t) - V/2]R(t) - 2[q d (t)»(P(t)) - q b (t)P(t)} letting e - 0, we obtain 
< (1 + Ppeak) 2 + rfc + r 2 ax + 2?7 0 r max + + (1 + ri 2 d )A 2 mm r _ 1 T _ 1 

iminf - ^ > lim inf - ^ 

J — >oo 1 * — ' 1 — »oo 1 * — ' 

t=0 t=0 



'max 1 max i A^max 

+ VR(t) - VR*(t) + 2q d {t)[R*{t) - fi{P*{t)) + I*(t)- 
D*(t)] + 2q d (t)[D*(t) - VdR*(t)] + 2q b (t)[P*(t) - r(t)] 

+ 2q d (t)I(t) + (V + 2A max )(/3 + Mmax)io(t) 

Without loss of generality, let q d (0) = B d . We have the 
following cases: 

i) if D(t) > 0, then from q d (t + 1) = (q d (t) - fJ,{P{t))) + + 
R{t)-D{t) and I{t) = (fi(P(t))-q d (t)) + , I(t) can be strictly 
positive only when A(t) > B d . However, whenever D(t) > 0, 
q d (t+l) = B d . 

ii) if D(t) = 0, from q d (t + 1) = q d (t) - n(P(t)) + I(t) + 
R(t) - D{t) and q d (t + 1) = (q d (t) - VdR(t)) + + D(t) - 
H(P{t)) + I(t) + R(t) - D(t), q d (t) decreases no slower and 
increases no faster than q d (t). 



-o(^)-o( { 



B d ) + 



+ (3) 



V 



)• 



IV) B d < oo and B b < oo. 

Simply by combing case II) and case III), we obtain 

T-l T-l 

lim inf - ]T R(t) > lim inf - ]T R*(t) - Vo {^ x + /?) 



t=o 



t=o 



0( 



y 2 



B d )+, 



B b )+ 



V 



)- 



V 
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C. Proof of Corollary 1 

Similar to the proof of Proposition 1, we have the fact 
if q b (t) is strongly stable, then limsupy^^ < 0. 

From Equation (13), we have qg(t + 1) > q b (t) - rf^ + 

Eien n u e„ P M - »V»(t) + M « (*) + '"(*)• Note that 9? (* + 
1) = <??(*) -£i 6 n B ue„ fl(*)+r n (t)-M n (t). By summing 
from 0 to T — 1, dividing by T and taking limsup of both 
sides, we have 



q n (T) 
lim sup — > lim 

T ~ T^oo T 



C(0) 



T-l 

C + liminf-^7^) 

7 — *oo / — 



Since limsup T ^ c 
lim sup T . 



+ lim 

97 (T) 



C(0) - <OT 



T 



cm 

->oo T 



< 0, lim sup T _ 



t=o 



5?(0) _ 



= 0 

0, and liniT^oo = 0, so we get 
K - lim sup^ 1 EL'o 1 7^ (t) < ^ , VneN. ■ 

Z). Proof of Theorem 2 

Proof of Equation (15) and Equation (18): We prove Equa- 
tion (15) by induction. Let q d ' dx (t) be the maximum data queue 
length for all flows at slot t. Assume that g™ x (t) < \ + A m . dx 
(holds for t = 0 by letting ^' e (0) = 0, Vn, e e M), need 
to show that it holds at slot t + 1. Consider the data queue 
<7^' e (i + 1) maintained at any node n for flow destined to any 
node e^nat slot t+1. If node n received data destined to 
e from other node m at slot t, then by the routing policy in 
Section IV-A and definition of u>( mi „)(£), q™' e {t) — q d ' e (t) > 
jf mn y where (m, n) is the link from node m to node n. 
Choose 7 such that the resulting backlog of the receiving node 
is no longer than that of the transmitting node (let ^JJJ ax to be 
the maximum endogenous arrivals, then 7 = /i" ax + A max 
satisfy this condition), we then have q^ ,e {t+l) < q d l ' e (t) + j, 
then q y{t + 1) < q™- e {t) < < f + ,4 max . If node n 

did not receive any data destined to e from other nodes, then 
it can only have exogenous arrivals. Clearly q d ' e (t + 1) < 
q^' e (t) < \ + A max if there were no exogenous arrivals. If 
there were exogenous arrivals, by AC of Section IV-A, we 
must have q d ' e (t) < \, then q d ' e (t + 1) < ^ + A max . Thus, 
Equation (15) holds. Equation (18) can be shown using the 
same argument. 

Proof of Equation (16) and Equation (19): Since 

MPi(t)) < w(o) + m(t) for Pt{t) e n,(t), vt > o, 

«;«(t)w W)) - (9T! n(,) (*) + C C( ° W) < ^WW(O) + 



^4max) for all t. This holds for all I E XL, so Equation (16) is 
proved. Equation (19) can be shown using the same argument. 

Proof of Equation (17): Define L(q d (t) , q b (t)) = 
E„, e (<?rW) 2 + £JTO) 2 , then 



A(t) = L(q d (t + 1), f 6 (i + 1)) - L(q d (t),^ b (t)) 

< J2i A Lx + 2A max ^ max + 2^ ax ] + ^[(1 + 

-Ppeak) + 



(% n ) 2 + r-Lc + 2r? 0 r ma x] + 2 £ <# e (t)i£(i) - 2 £ 

'(^«»(').«»(*) (t) _ g rec(0,e,(t) (t))w(fl(t)) _ ( -tra»(i) 

(t) + ^ C(,) (t))fl(t)] + 2^ «?(*) [TO - r n (*) + 

n 

M„(t)] 



x , , .1 . tran(l),ei (t) / .\ rec(l) .ei (t) / . \ , ei (t) 

Note that if q d w (*) — 5<j (*) < 7; > 

then = 0 and /U;(P;(i)) = 0, then 



(9d 



tran(l),ei (t) 



otherwise, g *™"«> e <«( t ) _ ? - c( /),e !( t) (f) > then 

(«r n(I) ' e,(t) (*)-<zr (0,e,(t) (*))w(^w) > ^ww(^w). 

We then have 



A(t) < + 2A maxAimax + 2 M Lx] + + ^ P eak) 2 + 

n,e n 

(€) 2 + rLx + 2r? 0 r max ] + 2 ^ ^(*)[i~(t) - r n (t)+ 

n 

M n ( t )] + 2Y,c e m e n (t) - 2^ K(t) W (fl(*))- 

(C a " (0 (*) + C cW (*))^w] 



/3 Wi (i)fl(i) - (C° n(0 (*) + C eClij )fl(^) where fl(t) is the 
solution of Equation (14). If /3wi(t)Pi(t) - {qf an{l \t) + 

€ c c { p m < °' then we get - (9T Q " (i) (^) + c eci ; j (mm - E J6 M( t )[^(t)w(ir(t)) - («r n ^(*) + 

ql ec ^)Pi(t) < wi(t)m(0). However, solution of Equa- 
tion (14) chooses Pi(t) that maximizes wi(t)fii(Pi(t)) — 
(qT an(l \t) + q r b ec(l) )Pi(t) which means «;,(*)/*« (^(*)) - 

( -tran(0 w+ -rec(0 )PiW > ^(^^(0) since Q g n ,(t). Then 

we must have M(t)P;(i) - + ^'^(t) > 0. 

Using a similar argument as in the proof of Equation (10), 
we obtain + q r h ec(l) ) < (3 Wl (t) < < /3(| + 



rec(Z)\ 



Note that Ez^HWwW)) (^ ron(0 (t) 



' ~tran(l) , 



q b (t))Pi(t)], where M(t) is the matching chosen by 
MWM algorithm and M* (t) is the matching picked by optimal 
policy in slot t, and Pf(t), V7 e XL are the suppositionally 
calculated power in MPA Since the objective function of 
power allocation component is separable over links and for 
each link it is concave, using similar argument as in single 



12 



link case, we have where -D™' e (i) is the amount of overestimated endogenous 

A M,VU2 +94 „ +9, ,2 IxUup ^, arrivals to the q ueue iT since E/ e e n MfW)) may be 

^ ^ — "I" z/1 maxf I max t z MmaxJ ' *pedk) i" larger than the actual endogenous arrivals; D^(t) is the 

™' e " amount of overestimated departures since Eien„ MfC^U*)) 

(r?™) + r max + 2?7 0 r max + (V + 2A max )(/3 + " max )7 0 ma y b e i ar g er than the actual departures. Further, at slot t, 

„, e V pe*t^\ 2 ^ nOC ' e n ^ oeS n0t ^ ave enc l 0 g enous arrivals for flow e, then 

Id \ ) 2 « w D^ e (t) = 0; if flow e is transferred from node n to node m, 

E m then L>™ u ' t 6 0) = D™' e (t). By choosing 7 = A m . dx + yu max , the 

2 , W ~ ?d W — 1)^1 \ r i \ t )) + 2 resulting backlog of the receiving node is no longer than that 



n,e ieO„ 



of the transmitting node. We then have 



£ i?(t)-r n (t) + A£(t) + M n (t) 

< E^™* + 2(^ m ax + 7)Mmax + 2^Lj + E^ 1 + ^ = E ^'"W 



n,e 

) 2 + Wof + r L + 2?7 0 r max + (V + 2A max )(/3 + 

/^max J 

■ to] +^E«w - co) + 2 E^ (*)M n (t)+ 



E«rwr(t+i)-9rw) 
- E 



zen„ 



n,e 

+ 2E^ n w 



K(t) + ( E-E Uoto) 



>E «?"(*) 



zeo„ U e„ 



(25) 



Since g^' e (t) - (j^ e (t + 1) < Anax + Mmax, Vi > 0, Vn, e e 
N and g^ e * are strongly stable Vn, e€N, then 

W ^ E E «?"(*)(«?"*(* + 1) - ?r (*)) 



Note that tf*(t + 1) = - Ej 6 n„u e„ + 

r n (t) — M*(t) for the optimal policy. By multiplying 
qg(t) for both sides and rearranging terms, we 

obtain q%(t) [e, g o„ U e„ TO ~ r n (t) + M*(t) 

Qb (*)[«&*(*) " C(* + !)]• B y summing from 0 to T — 1, 

dividing by T and taking lim inf 00, we have 



^E 

71, e 



t—0 n.e 

n.e (rp\ n,e* (rp\ 

lim ^ ^ (7) +(A max + A , max ) 



T 



?r(o)«r*(o) 



• lim inf — q\ ' e * (t) — lim 



T-l 



Elim inf 



liminf lEE 9?* W(9T(*) - #(* - 1)) 



E Pi*(t)-r n (t)+M* n (t) 
ien n u e„ 



< 



E 



(ir + A max ) lim ^ ( + (A m . dx + li^) 
Z 1 — ^00 1 



9?(o)c(o)-s(vm 



+ 



L = l 



1 T 

< E( A «- + ^ l ^ ini r E c e *(t) 

— _/ — >oo / — 



t=l 



t=i n which is a finite constant that is not related to V. 

! t Thus, we have i liminf^^ i E n ,e C + 

<(1 + WS^^?? (i)<(l + P P eak)E^ n Ejee^f^W) -E, e n B Mf(i7(*))] < °(f)- Thus ' b y 



'- 1 n ™ summing from 0 to T — 1, dividing by T and V, taking 

The argument of case II) in the proof of Theorem 1 can be liminf T ^oo for Equation (25), we get 



directly applied here, and we have 

1 1 T_1 (@-V — R n \ + 

lhninflEEg(W)<E°( 2 / )■ 



T-l 



eI^^ee^) 



y t- 



v 



t= 0 n n 

Note that the data queue dynamics can be written as 

c e (t+ 1) = q n d e (t)+K(t) + E ^f(^(*)) - E 
/if(^(*))-£>r (*)+«(*). 



1 



t=0 e 
T-l 



^ E l ™} n{ tEE ^ (*) - ^"(^max + /?) 



+0( 



t=0 e 



V 



) -O(-). 



which is Equation (17). 
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Proof of Equation (20): Define L(q d (t) , q b (t)) = 
lEnM^^ + hEnm 2 , and then 

A(t) = L(q d (t + 1), f 6 (i + 1)) - L(q d (t), &(t)) 

< E^max + (Anax + T^max + Mmax] + ^ ^[(1+ 

n,e n 

Ppeak) 2 + (r?" ) 2 + rl dx + 2ry 0 r max + (V + 2A max )(/3+ 



and + 1),.... Thus, l [leM{t)] and nf(P^(t)) are 

independent and then uncorrected, so 



T-l 



'[l£M(t)] 



t=0 

1 T-l j . . 



T-l 



t=0 

/w)^(*)] + ^E K(t) + \ Ei?r w y\K*{t) = liminf i ^ _ e , 7 



t=0 



6M{t)] • 



n 



E P,*(*) -»■„(*) + M^(t) 
iefi„ u e„ 



E - E rt e (prw)^ieM( t) ] 

\iee„ zen„/ 



+ 



E9f(*)M n (t). 



Here the difference from the MWM case is that we 
need to show £ liminf T ^ £ ^f=o E„ e ^WIl^M + 

£i f e B /W(*)) J [JeM(t)] ~ E Ie n„ A*f(>r (t))i[JeM(t)]] < 
O(y). Similar to the MWM case, we only need to show that 
given that all the data queues are strongly stable under the 
optimal policy queue evolution, i.e., 



9 r*(t+i) < (#"*(*)- E tf(Pi*(t))) + + 



;ee„ 



we have all the corresponding data queues strongly stable with 
the following evolution, 

«T*(t + 1) < (qY*(t) E (*7(*))J[leM(t)]) + + 

ien n 

1 -K(t)+ E (27) 

Let iJ* = {-ff^* e } denote the routing matrix under the 
optimal policy, where H l * e = 1 means the sensing data at 
node n destined for node e is routed through link I. Let 
fif* = liminf T ^oo ^T,J=o Vi( p *(t))- Then Equation (26) 
implies that all the accumulative data queues with the follow- 
ing queue evolutions are all strongly stable: 

QY*{t + l) < (Q d e *(t)- E + 

ien n 

E™6N Ee R m (*)#m,e 



(28) 



Note that hi^Mlt)] is on ly related to the state of data 
queues at slot t, which is a function of r(0), . . . ,f(t — 1) 
and A(0),A(l),...,A(t - 1). P t *(t) is the optimal power 
allocation decision which can be obtained from backward 
dynamic programming, so it is a function of f(t), f(t+ 1), . . . 



t=o 



In order to show Equation (27), we only need to show all 
the accumulative data queues under the MM based algorithm 
with the following queue evolutions are strongly stable: 

Q n d e *(t + 1) < (Q n d ' e *(t) E hieM( t)] ) + + Kit) 

ien n 

+ E • (29) 

iee„ 



Equation(28) implies Equation (29) by the existing result 
from [15]. ■ 



